Interactivity and Large-scale Viewing in 3d Tele-immersion By

Over the last few years, we have seen a surge of interest in telepresence video collaborative technologies where remote users perform virtual-reality-like interactions with each other using real-time 3D video. Though the initial focus was on video conferencing applications, recent developments in audio-video and network technologies have expanded the horizon by supporting full-body interaction for many other physical activities in the virtual environment. Despite great potential, 3D tele-immersive (3DTI) sessions still face significant challenges due to high interactivity, multi-stream dependency, multi-view dynamism and large resource demand. Quality of service (QoS) allocation in 3DTI sessions not only depends on the availability of network and processing resources, but also on the users’ expectations, which are dynamic and heterogenous. In this dissertation, we revisit the design space of 3DTI session management framework. We propose a novel, comprehensive, user-experience-aware, programmable, cross-layer session control framework for 3DTI applications. Our control framework consists of a distributed monitoring oracle, a decision engine for computing userexperience-aware multi-stream dissemination topology, and a controller for allocating resources and configuring application and network layer data plane at the users. The monitoring oracle allows run-time session queries by arranging application and system layer metadata over a tree-based overlay. It resolves multi-attribute range-based performance queries in real-time without interfering application data traffic. Based on the monitored resources, performance queries, and user demands, we compute a multi-stream content dissemination topology and allocate QoS among the users. Due to the differences in interactivity requirement and scale, we consider separate content distribution solutions for immersive users (who interact with each other in the shared virtual environment in real-time) and non-immersive users (who watch the collaborative activities performed by the immersive users, but do not immerse themselves into the virtual world). The number of immersive users are small, however, the number of non-immersive users can be in the order of thousands. The immersive user prioritizes interactivity and drops delayed streams to ensure consistency across multiple geographically distributed streams. QoS allocation in this domain influences 3DTI experience of the users based on the on-going 3DTI activity. Different activities require different performance QoS profiles to ensure strong quality of experience (QoE) of the users. We model the problem of multi-stream QoS allocation as a prioritized multi-objective optimization problem, and solve it using an evolutionary approach. We show that the prioritized optimization of QoS meets expectations of the users up to 50% higher compared to the existing content distribution